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Engineering  Systems  Thinking: 

•Definition 

•Assessing 

•Correlation  with  Project  Success 


Engineering  Systems  Thinking:  Definition 


Engineering  Systems  Thinking 

Systems  thinking’ is  a  discipline  for  seeing  wholes  (Senge,  1994). 


Engineering  Systems  Thinking  is  a  major  high-order  thinking  skill 
that  enables  individuals  to  successfully  perform  systems 
engineering  tasks. 


To  successfully  perform  systems  engineering  roles,  systems 
engineers  need  a  systems  view  or  a  high  Capacity  for  Engineering 
Systems  Thinking  (CEST). 


It  was  found  that  this  ability  is  a  consistent  personality  trait,  and 
that  it  can  be  used  to  distinguish  between  individual  engineers. 


Engineering  Systems  Thinking:  Assessing 


Assessing  Systems  Thinking  in  Engineers 


Assessing  CEST  (Capacity  for  Engineering  Systems  Thinking) 


Frank  (2010)  introduces  an  interest  inventory  for  assessing 
engineers'  interest  regarding  systems  engineering  positions 
and  the  results  of  three  studies  aimed  at  examining  its 
reliability  and  validity. 


The  will  and  the  interest  to  be  a  systems  engineer  mainly 
means  the  will  and  interest  to  deal  with  positions  that 
require  a  capacity  for  engineering  systems  thinking  (CEST). 


Interest  inventory  is  a  very  common  tool  which  is 
frequently  used  to  help  people  choose  a  profession,  and  as  a 
selection  tool  (to  determine  whether  a  certain  individual  is 
suitable  for  a  certain  role)  in  the  recruiting  process. 


Assessing  CEST  (Capacity  for  Engineering  Systems 


The  content  validity  of  the  interest  inventory  was 
achieved  by  basing  its  items  on  the  findings  of  a  prior 
study  aimed  at  identifying  the  characteristics  of 
successful  systems  engineers  (Frank,  2006). 


Thirty-one  competencies  of  successful  systems 
engineers  were  found  in  this  prior  study  and  they  were 
classified  into  ten  cognitive  characteristics,  eleven 
abilities,  ten  individual  traits  and  three  dealing  with 
multidisciplinary  knowledge  and  experience. 


Engineering  Systems  Thinking: 
Correlation  with  Project  Success 


According  to  the  Standish  Group  Report  (2009), 

68%  of  all  projects  failed: 

❖44%  of  the  projects  were  late,  over  planned  budget,  and/or 
had  less  than  the  required  features  and  functions. 

❖  24  %  were  cancelled  prior  to  completion  or  delivered  and 
never  used. 

What  causes  projects  to  fail?  Many  reasons  can  been 
found  in  the  literature. 

We  focus  here  on  one  reason  -  lack  of  engineers  with 
a  high  capacity  for  engineering  systems  thinking 
(CEST). 
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The  study  aimed  at  exploring  the  relationship 
between  ( 1 )  systems  engineers'  capacity  for 
engineering  systems  thinking  (CEST),  (2)  project 

types  and  (3)  project  success. 


We  have  already  defined  CEST  (slide  4). 
Let  us  now  explain  what  do  we  mean  by: 

•  Project  types 

•  Project  success 


One  of  the  common 
misconceptions  is  that  similar 
tools  can  be  used  for  all  projects. 

Shenhar  and  Dvir  (2007) 
identified  four  dimensions  to 
distinguish  among  projects: 
Novelty,  Technology/uncertainty, 
Complexity,  Pace  (NTCP  model). 

NTCP  model  can  guide  project 
managers  and  systems  engineers 
in  selecting  their  project  handling 
style. 
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The  literature  traditionally  use  time,  budget,  and 
performance  as  the  main  indicators  of  project  success. 

More  elements  to  the  assessment  of  project  success 
found  such  as: 

•  Stakeholders'  satisfaction. 

•  Efficiency  of  the  implementation  process. 

•  Personal  growth. 

•  Business  and  financial  performance. 

•  The  creation  of  new  opportunities.  Time 

•  (and  many  more  . . . ). 


The  Iron  Triangle 
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A  self  report  questionnaire  comprised  of  three  parts  : 

Assessing  CEST 

•  This  tool  developed  by  Frank  (2010)  -  slides  6  and  7. 

Assessing  project  success 

•  Measured  along  5  dimensions:  Meeting  planning  goals 
(project  efficiency),  customer  benefits  (success  from  the 
customer’s  point  of  view),  benefits  to  the  developing 
organization,  benefit  to  the  community  and  national 
infrastructure,  benefit  to  the  project  team. 

Identifying  project  type 

•  The  items  in  this  part  were  based  on  the  NTCP  model. 
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Population  -  all  senior  systems  engineers  employed  in 
the  'high-tech  -  electronics  -  systems'  industry  in  Israel 
(including 

Sampling  frame  -  all  senior  systems  engineers 
employed  in  the  sixteen  largest  'high-tech  -  electronics 
-  systems'  companies  in  Israel. 

Sample  -  1 1 4  senior  systems  engineers  who  were 
randomly  selected  from  the  sampling  frame  (sampling 
error  9.18%,  p<0.05). 

•  About  40%  of  the  survey's  participants  were  from  the  defense 
industry. 


14 


The  projects  can  be  classified,  according  to  the  NTCP  model,  as 
follows: 

Novelty:  54  subjects  were  engaged  in  ’platform’  projects,  30  in 
’derivative’  projects  and  30  in  ’breakthrough’  projects. 

Technology:  11  subjects  were  engaged  in  ’super-high-tech’  projects, 
65  in  ’high-tech’  projects,  26  in  ’medium-tech’  projects  and  12  in 
’low-tech’  projects. 

Complexity:  44  subjects  were  engaged  in  ’array’  projects,  60  in 
’system’  projects  and  10  in  ’assembly’  projects. 

Pace:  3  subjects  were  engaged  in  ’blitz’  projects,  23  in  ’time-critical’ 
projects,  41  in  'fast-competitive’  projects  and  47  in  ’regular’ 
projects. 


The  duration  of  the  projects:  6  months  -  3  years. 
The  budget:  $200K  -  200M  (average  -  $28. 7M). 
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Two  stages: 

Pilot  survey  -  36  senior  systems  engineers  participated. 
Main  survey. 

The  findings  of  the  pilot  study  were  used  to  revise  and 
improve  the  questionnaire. 

Confidentiality  at  all  stages  was  promised  and  enforced. 
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Main  Results  (slide  1  of  5) 
CEST  Scores 


CEST  Group 

Mean 

N 

Std.  Deviation 

1  (Low  CEST) 

46.94 

9 

9.745 

2  (Medium  CEST) 

72.28 

45 

4.909 

3  (High  CEST 

84.67 

60 

5.094 

Total 

76.80 

114 

11.902 

Main  Results  (slide  2  of  5) 

Correlations  between  the  subjects4  CEST  and 
the  projects'  five  success  criteria 


Efficiency 

Custor 

Team 

Business 

Future 

PRSucc 

Pearson 

Correlation 

^249(**)) 

.065 

.050 

(338(**V 

^305(**)^ 

^310(^) 

Sig.  (2-tailed) 

.008 

.503 

.601 

.000 

.001 

.001 

N 

114 

109 

114 

105 

114 

114 

The  findings  indicate  that  there  is  a  positive  significant 
correlation  between  subjects'  CEST  and  project  success  in  four 

dimensions 
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An  ANOVA  test  was  performed  to  examine  whether  the  project  type 
(according  to  the  NTCP  model)  is  a  moderator  variable  that  affects 
the  correlation  between  the  subjects’  CEST  and  project  success. 


It  was  found  that  the  project  type  does  not  significantly  affect  the 
correlation  between  the  subjects’  CEST  and  project  success. 


However,  in  order  to  test  whether  there  is  a  specific  dimension 
(novelty,  technology,  complexity  and  pace)  that  affects  the 
correlation  between  the  subjects’  CEST  and  project  success,  four 
additional  two-way  ANOVA  tests  were  performed  -  one  test  for  each 
dimension. 
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It  was  found  that  the  variable  'novelty'  does  significantly  affect 
the  correlation  between  the  subjects'  CEST  and  project  success. 


Post-Hoc  tests  revealed  that  the  more  innovative  the  project, 
the  higher  the  correlation  between  the  subjects'  CEST  and 
project  success. 

In  other  words,  successful  systems  engineers  (systems 
engineers  with  high  CEST)  are  needed  most  in  platform 
projects  (projects  that  produce  a  new  generation  of  products) 
and  breakthrough  projects  (radical  innovative  projects). 
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3,111  RCSllltS  (slide  5  of  5) 


No  significant  correlation  was  found  between  CEST 
and  the  satisfaction  level  of  the  projects'  teams. 


No  significant  correlation  was  found  between  CEST 
and  the  satisfaction  level  of  the  customer  and  end- 
users. 


Organizations  that  pursue  customer  satisfaction 
should  nominate  projects  managers  who  are 
committed  to  success  in  these  measures. 
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inclusions  (sli&TFoF^J 


The  findings  of  this  study  clearly  show  that  there  is  a 
significant  correlation  between  CEST  and  project  success. 

The  extent  of  the  project's  novelty  (derivative,  platform  or 
breakthrough)  is  a  moderator  variable  that  affects  this 
correlation. 

The  more  innovative  the  project  is,  the  higher  the  correlation 
between  the  subjects'  CEST  and  project  success. 

Successful  systems  engineers  (systems  engineers  with  high 
CEST)  are  needed  most  in  platform  projects  (projects  that 
produce  a  new  generation  of  products)  and  breakthrough 
projects  (radical  innovative  projects). 
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nchisioiis  (slid<T2~oi 

However,  the  findings  of  the  current  study  show  that 
the  coefficient  of  determination,  R2,  is  relatively  low. 

This  means  that  the  prediction  of  project  success  can 
be  only  minimally  based  on  CEST. 

Only  a  low  percent  of  the  variation  in  project  success 
can  be  explained  by  CEST.  The  remaining  percentage 
should  be  explained  by  other  variables. 

Of  course,  this  finding  makes  sense,  as  many  other 
variables  might  explain  project  success, 
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inclusions  (slideToF3j 


In  any  case,  a  significant  correlation  between  CEST  and  project 
success  does  exist. 

Because  correlation  is  necessary  for  causation,  it  is  clear 
beyond  all  doubt  that  organizations  should  select  engineers 
who  possess  a  high  capacity  for  engineering  systems  thinking. 

Organizations  also  should  create  a  supportive  environment  for 
enabling  systems  thinking  development  in  engineers. 

But,  what  are  the  best  ways  to  create  such  a  supportive 
environment?  Is  the  engineering  systems  thinking  capability 
acquired  or  innate? 
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Previous  studies  show  that  CEST  can  be  developed 
through  experience  and  learning. 

Therefore,  organizations  should  create  a  supportive 
environment  for  enabling  systems  thinking 
development  in  engineers  and  managers. 

Engineers  and  managers  with  a  high  CEST  may  lead 
to  better  performance  in  general,  and  especially  in 
regard  to  meeting  design  goals  and  overall  project 
success. 
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Any  questions? 
THANK  YOU 


Thank  You! 


